Multiple choice decision engine for an electronic personal assistant

ABSTRACT

To simplify assisting a user in their day-to-day activities, a communication for performing an action may be sent to a user in the form of a query, where the query includes the most likely set of choices for the action arranged in a group of dichotomous (e.g., yes/no) or multiple choice answers. In this manner, a user may respond to the query by simply selecting one of the dichotomous or multiple choice answers. Historical logs of past actions, responses, queries, and so forth, may be used to predict future user actions or needs, and to formulate future queries for sending to the user. These techniques may be implemented, for example, through a remote coordination server or directly through a user&#39;s personal electronics device.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.13/358,004 filed on Jan. 25, 2012, which claims the benefit of U.S. App.No. 61/435,985 filed on Jan. 25, 2011.

FIELD OF THE INVENTION

The present invention is in the field of artificial intelligence. Morespecifically, the present invention is in the field of electronicpersonal assistants using adaptive intelligence to evaluate likelyactions and decisions relevant to the user for the purpose of assistingthe user to organize his or her day-to-day activities.

BACKGROUND OF THE INVENTION

People generally have a need for organizational tools to manage theirbusy lives. Such tools have, in the past, consisted of clumsypaper-based systems, and more recently have begun to make use ofcomputer technology. Today, portable personal assistant devices, such assmartphones and personal digital assistants (PDAs), exist to help peoplekeep track of information, scheduling, appointments, reminders, contactinformation and myriad other bits of data. Such devices can presentprescheduled alerts and reminders to the user. Many of the latestdevices can synchronize their data with remote servers, and receivemessages and instructions from remote sources. However, these devicesgenerally do not have any sort of built-in intelligence. They are unableto adapt to the behavior, environment or needs of the user. The user hasto explicitly instruct the device to perform any specific action. Thislimits the usefulness of such devices as personal assistants, as theuser must constantly monitor the function of the device to ensure it hasall the correct data and has been programmed to perform the correctactions at the correct times.

It should be noted that Patent 61/445,433 includes concepts related tothe invention described herein.

Hands-free computing devices, such as Bluetooth headsets, exist thatallow the user to interact with a computer or mobile device, givingcommands and receiving information. Rudimentary neural interfaces evenexist that can detect commands and responses indicated by the user'sthoughts and emotions. However, none of these hands-free devices iscapable of connecting directly and immediately to a more distributedsource of processing power, such as a “cloud” of networked devicesavailable for computing, and none is capable of adapting its behavior tothe user's current circumstances—environment, status, activity, etc.—orpredicting the user's desires and proactively consulting the userregarding potentially desirable actions and information.

In using a neural interface, the user thinks a particular thought orexpresses a particular emotion to register a particular input, such as“yes” or “no.” An example of such a device is the NeuroSky MindSet.These neural devices, however, can be difficult to learn to use, theyrequire a great deal of concentration to ensure the correct thought isbrought to mind or the correct emotion expressed, and they are quiteslow in terms of how quickly the user can register an input. The levelof concentration necessary precludes use of a neural interface whileperforming other tasks requiring concentration, such as driving a car,holding a conversation or operating machinery. Further, the difficultyof registering an unambiguous stimulus to the interface makes thesedevices useful only for the simplest and least critical applications.These are severe limitations that prevent many potential uses for neuralcontrol devices.

Transcription services exist whereby a user can call a specifictelephone number, speak a message and have the message sent to any of anumber of destinations, including an email address, a blog and themicroblogging service Twitter. The limitation of these services,however, is that the user must call a number and wait for a connectionbefore speaking his or her message, which consumes a great deal of extratime and requires extra focused attention.

Predictive dialers exist that automatically call telephone numbers, playa recorded message, wait for a response from the recipient of the phonecall, whether via keypad entry on the telephone or via interactive voiceresponse (IVR), and then play another recorded message or transfer therecipient to a particular telephone number based on the response. Butthese dialers are fixed in their functionality, cannot ask questionsautonomously and cannot automatically adapt to the user's preferencesand behavior.

Limited data prioritization systems exist, such as Gmail's PriorityInbox, that adaptively sort information based on relevance orimportance. However, these systems are generally limited to specificdomains and do not operate broadly enough to be useful in a truepersonal assistant system.

Numerous services exist that present recommendations (such as productrecommendations) based on historical and “crowd-sourced” data—datagathered from the behavior of large groups of users—such as Hunch.com,Mint.com and Amazon.com. However, none of these services issophisticated enough to support a true personal assistant system.

Systems have been previously described that estimate a user's current“state” in terms of activity and availability, but none of these systemsuse the resulting conclusions to provide the adaptive interaction anddecision-making capabilities of a true personal assistant. Further, somesystems make possible semi-automated scheduling using some level ofautomatic decision-making capability, but these systems are limited inscope and fall short of the qualities desirable in a true electronicpersonal assistant. Other systems exist that make use of distributeddata sources and the user's current context to provide relevantinformation, but these too fall short of a true personal assistant inthat they simply provide information and do not provide thedecision-making capabilities desired of a personal assistant.

Live, remote personal assistant services exist that provide thefunctions of a personal assistant to many customers for a fee, andoperate via phone, email or other communication channels. However, theseservices are just a different way of providing traditional personalassistant services, and still require that a live individual perform theservices. The automation of services is not present, as is desirable ina true electronic personal assistant system. Further, as the customerusually does not know the hired assistant personally, lack of trust andclose communication can present obstacles to effective service.

When being called on to make decisions, executives usually prefersimple, clear proposals, providing all relevant data and requiringapproval or disapproval (or possibly deferral). This reduces the burdenon the executive and increases efficiency. However, existing digitalassistant devices fail to meet this need, requiring a human operator tofind, evaluate and analyze information. Thus, the executive either hasto do the work himself or hire a live assistant to manage his digitalassistant, effectively defeating the purpose.

The present invention has the object of solving these problems.

SUMMARY OF THE INVENTION

It is an object of the present invention to provide an electronicpersonal assistant device that assists the user in recalling informationand provides the user with timely reminders and alerts.

It is another object of this invention to query and/or infer the user'sbehavior patterns, on a one-time or ongoing basis, so as to be able topredict desired or necessary actions and present them to the user forapproval.

It is another object of this invention to record the user's behavior andobserve the user's current location and circumstances, and use theresulting data to adapt its operation to best suit the user's needs,predicting as reliably as possible which data the user will need at anygiven time and presenting it to the user.

It is another object of this invention to ask the user dichotomous ormultiple-choice questions—i.e., requiring yes/no or other dichotomousanswers, or questions with a larger number of choices—to determinewhether predictions made as to the user's needs are correct, and adaptfuture behavior based on the responses to such questions.

It is another object of this invention to reduce decisions to simple,direct queries that can be presented to the user, briefly and with allrelevant data, for decision, such that the user is asked to make asimple choice between two or more options.

It is another object of this invention to allow remote storage andprocessing of the user's data, including execution of the adaptation andprediction algorithms used.

It is another object of this invention to allow the remote storage andprocessing unit to initiate dichotomous queries of the user bycommunication via the user's electronic personal assistant device, andreceive the responses to said queries from the device for storage andprocessing, via a phone call, computer network communication or othermethod.

It is another object of this invention to allow the user to delivercommands to an electronic personal assistant device, via speech oranother means, for execution, either directly on the device or via theremote storage and processing unit.

It is another object of this invention to provide the user with avoice-driven interface allowing control and communication solely viaspeech, but also to provide a physical interface for control andcommunication with the system.

It is another object of this invention to detect, either by direct usercommand or by recognition of unique environmental circumstances, the“state” of the user—i.e., whether the user is busy and shouldn't beinterrupted, needs only urgent alerts and reminders, should be freelycommunicated with, etc.

It is another object of this invention to provide a neural interface,using the brain's emitted waves, neurally induced energy effects,muscular impulses or physical movements to detect dichotomous (e.g.,yes/no) or simple multiple-choice answers to questions.

It is another object of this invention to make possible variousdifferent types of user interfaces, including, but not limited to,neural, emotional and physical interfaces.

It is another object of this invention to provide a low-latency userinterface that responds promptly even when using a remote unit forstorage and processing, and that is “always on” and will react quicklyto voice or other commands without any sort of initialization or wake-upcommand.

It is another object of this invention to provide a low-latency userinterface that responds reliably, and will either deliver the userrequest or promptly inform the user that the task will be delayed untilthe necessary resources (such as network connectivity or a functionalsmartphone) are available.

It is another object of this invention to provide an active taskmanagement and reminder system to assist the user in accomplishingdesired tasks in specified time frames.

It is another object of this invention to provide a personal contactservice wherein a meeting with one or more other parties can bescheduled and initiated without direct action by the user, via anyavailable communication channel.

It is another object of this invention to allow integration with livepersonal assistant services and to provide an abstraction layer for suchservices.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 is a diagram of the components and interconnection of oneembodiment of the personal assistant system described herein.

FIG. 2 is a diagram showing the “always on” aspect of a communicationsinterface device as used in some embodiments.

FIG. 3 is a flow chart for the implementation of the “always on” aspectof the system according to one embodiment.

FIG. 4 is a diagram showing the connectivity of the system to the user'svarious data stores.

FIG. 5 is a diagram showing examples of possible system configurations.

FIG. 6 is a diagram depicting the use of auxiliary “plugin” modules toexpand the functionality of the system.

DETAILED DESCRIPTION OF THE INVENTION

Illustrative embodiments and exemplary applications will now bedescribed to disclose the advantageous teachings of the presentinvention.

While the present invention is described herein with reference toillustrative embodiments for particular applications, it should beunderstood that the invention is not limited thereto. Those havingordinary skill in the art and access to the teachings provided hereinwill recognize additional modifications, applications and embodimentswithin the scope thereof, and additional fields in which the presentinvention would be of significant utility.

Some portions of the detailed descriptions that follow are presented interms of algorithms and symbolic representations of operations on datawithin a computer memory. These algorithmic descriptions andrepresentations are the means used by those skilled in the dataprocessing arts to most effectively convey the substance of their workto others skilled in the art. An algorithm is here, and generally,conceived to be a self-consistent sequence of steps leading to a desiredresult. The steps are those requiring physical manipulations of physicalquantities. Usually, though not necessarily, these quantities take theform of electrical or magnetic signals capable of being stored,transferred, combined, compared and otherwise manipulated. It has provedconvenient at times, principally for reasons of common usage, to referto these signals as bits, values, elements, symbols, characters, terms,numbers or the like.

It should be borne in mind, however, that all of these and similar termsare to be associated with the appropriate physical quantities and aremerely convenient labels applied to these quantities. Unlessspecifically stated otherwise as apparent from the following discussion,it is appreciated that throughout the description, discussions utilizingterms such as “processing,” “computing,” “calculating,” “determining,”“selecting,” “displaying,” “sending,” “receiving,” “updating,”“modifying,” “assigning,” “requesting,” “notifying,” “communicating,”“interfacing,” “encrypting,” “decrypting” or the like refer to theactions and processes of a computer or similar electronic device havingprocessing circuitry and components that manipulate and transform datarepresented as physical (electronic) quantities within the system'sregisters and memories into other data similarly represented as physicalquantities within the computer's memories, registers or other suchinformation storage, transmission or display devices.

As generally shown in FIG. 1, a preferred embodiment consists of apersonal electronic device 101, such as a smartphone or PDA; acommunications interface 102, such as a Bluetooth headset or neuralinterface; and one or more remote coordination servers 103. Thecommunications interface 102 serves as the user's primary means ofinteraction with the system, allowing the user 104 to issue commands,receive data and respond to queries from the system. The personalelectronic device 101 serves as a communication link to the remotecoordination server(s) 103 and other services, such as telephony andInternet services, but can also be used as a secondary means of userinteraction with the system. The remote coordination server(s) 103perform the bulk of the system's processing tasks and serve as thecentral management point for the user's data, although some of thesefunctions can be performed by the personal electronic device as itscapabilities permit. The personal electronic device 101 and thecommunications interface 102 are connected to the remote coordinationserver(s) 103 via a communications network 105, such as a computer ortelephony network.

In one embodiment of the present invention, the communications interfaceis a headset, such as a Bluetooth headset commonly used with mobilephones, and the user interacts with the system using voice commands andresponses. The headset is linked to the personal electronic device, inthis case a smartphone. The smartphone in turn connects to a telephonynetwork, the Internet and other such services, and uses telephony and astandard TCP/IP network connection to communicate to a set of remotecoordination servers with a standardized communication interface.

The user can issue a vocal command to the headset, which detects thecommand and forwards it to a software control program running on thesmartphone, headset or other device. A key feature of this embodiment isthat there is no need for the user to press a button or speak a commandto “wake up” the system and prepare it to receive commands. The headsetis always listening for user commands and detects when the user hasspoken an intelligible command that can be executed. This is illustratedin the embodiment shown in FIG. 2: a headset 201 is always listening 202for audio input 203 and processing any commands it detects. However, insome embodiments it is possible to signal the beginning of a verbalcommand by pressing a button on the headset or other communicationsinterface. In one embodiment, as shown in FIG. 3, the headset idles in alow-power mode 301 waiting for a sudden increase in volume 302, and thenrecords and processes the subsequent sounds, looking for a parseablecommand 303. If it finds such a command 304, it passes it to thesoftware control program running on the smartphone 305. This can be donein several ways. In one implementation, the entire recording of the usercommand is passed to the control program (buffered mode). In anotherimplementation, the moment a command is detected, the audio informationis streamed to the control program in a way similar to “cut through”routing, as is known in the art. In another implementation, the audio isalways being streamed to the control program, either continuously orwhenever an audio threshold is exceeded. If the smartphone is notimmediately available for some reason—e.g., its power is off or it isdisconnected from the headset, the headset stores the received commandin a buffer, waiting for the smartphone to become available again, atwhich point it proceeds to send all buffered commands to the softwarecontrol program running on the smartphone. In another embodiment, theheadset itself is doing the command parsing using a control program.Some examples of commands the user could deliver to the system are toinitiate a phone call; pick up a phone call; hang up a call; send apredefined message via email, text message, etc.; pay a bill using aspecified bank account; purchase a specified item; post a status updateto a social networking Web site; order a meal for pick-up from arestaurant.

The smartphone, upon receiving a command, either executes the commanditself or forwards it to one of the remote coordination servers forexecution. Some commands have no need for remote execution, such as arequest to initiate a phone call, and these are executed by thesmartphone itself. Other commands, such as a request for a Web search,data from a Web site or information based on the user's currentgeographical location, requires a connection to a remote service, andwould thus be routed to one of the remote coordination servers forhandling.

The remote coordination server then processes the request and returns aresponse indicating success or providing the requested data. Uponreceipt of this response, the smartphone displays a suitable message forthe user and additionally forwards the response to the headset, whichinforms the user vocally of the result if appropriate.

In a similar fashion, a remote coordination server can originatecommunication to the user. Based on its internal algorithms, the remotecommunication server may deem it necessary to provide the user withinformation or present the user with a query, in which case it sends amessage to the user via any available communication medium. It may placea phone call to the user and vocally deliver the communication, or itmay transmit the communication via the Internet to the user'ssmartphone, which in turn delivers the message to the user vocally or onits screen.

The user can configure the system to deliver notifications when specificevents occur, and the system will monitor the user's data sources toestablish when such events have occurred and deliver a notification viaany available communication channel (or as limited by the user'sindicated preferences) to the user. For example, the system can notifythe user when a particular person is in a particular location; when anyof the user's friends or family are nearby; when a particular personchanges location; when a particular person becomes available online, ason an instant messaging service; when the status of a house alarmchanges; when the status of a remote computer server changes (e.g., theserver stops responding); when a communication is received, such as avoicemail, text or email message; when the price of a security changessufficiently; when particular events occur in an economic market; when aparticular business condition is present; when some event on anassociate's calendar is changed; when an event is scheduled or changedon a user's own calendar; when an event occurs nearby that is relevantto the user; when traffic conditions change in the user's path; when anairline flight changes its schedule. These notifications, of course,require that the system have access to the relevant data, but thesystem, having access to all the user's data stores and sources, as wellas worldwide network access, has access to an immense amount ofinformation. The available information is analyzed based on the user'srequest for notification and based on behavioral information about theuser, as in the case of the general relevance of an event to the user.

In some embodiments of the present invention, the user can answerqueries via a standard neural interface, as is known in the art. Theuser simply dons the appropriate apparatus and thinks or emotes thedesired response to questions posed by the system, the answers beingdetected by the neural interface and passed to the user's personalelectronic device for handling.

In various embodiments of the system, different user interface devicescan be used, in place of or in addition to the headset specified for thepreferred embodiment. Such interfaces can include, but are not limitedto, neural interfaces, face-tracking systems, eye-tracking systems,gestural devices, bodily sensors, interactive voice response systems andvoice control devices. The user can register input via any of a numberof possible methods, including, but not limited to, voice or othersounds; pressing buttons on a keypad, as on a phone; thinking aparticular thought; expressing a particular emotion; tensing a muscle;typing on a virtual keyboard; blinking; moving the eyes; making aparticular face; smiling and/or frowning; winking; sighing; whispering;making a specific motion; raising a number of fingers; shaking ornodding the head; and exercising a “phantom limb,” wherein the useractivates a little-used muscle or nerve; muscular stimuli; or electronicsignals induced in the nerves, brain and surface of the head or body bythoughts or emotions felt by the user. Output to the user can employ anyof a number of methods as well, including, but not limited to, displayof data on a screen or other visual output device; audio cues, whethervocal or not; vibration; neural stimuli; music; or electronic wavesapplied to the body, head or brain that induce thoughts or emotions inthe user.

FIG. 4 shows a few of the possible system configurations. Note thatthese are merely examples and by no means encompass all possible systemconfigurations. As shown in FIG. 4, the user interface device could be aheadset 403/406, a wristwatch 404/407 or a vehicle 405/408. The userinterface could then connect directly to remote coordination servers viathe Internet 402, or could connect via a mobile phone 401 to the remotecoordination servers.

In a preferred embodiment, the system has access to any available datastores belonging to the user. Namely, the system can draw upon any ofthe user's data stored in the “cloud,” such as Web-based email, contactdatabases, data stored on social networking Web sites, online retailtransactions, news feeds, etc. This is illustrated in the example shownin FIG. 5, wherein the personal assistant server 501 has access to (andthus can retrieve data from and send data to) Facebook 502, Amazon.com503, Gmail 504 and 1-800-FLOWERS 505. The system and user interfaceprovide the user with immediate access to all available data stores bysimple command, and suggest relevant data as described below.

In a preferred embodiment of the present invention, the remotecoordination servers implement a degree of intelligence by adapting tothe user's behavior, location and circumstances, and attempting topredict the user's needs. The remote coordination servers have access tothe user's present location via the facilities of the user's personalelectronic device (e.g., a smartphone), and use this location inconjunction with the time and other current data about the user todetermine what data might be relevant to the user, such as informationabout nearby businesses, proximity of associates or friends, tasks theuser needs to perform near the current location, etc. In addition, theremote coordination server maintains an historical log of the user'sactions in using the system for the purpose of discerning patterns andpredicting potential future user actions or needs. Further, the systemcan use an aggregate historical log of all its users to establishcorrelations between users for the purpose of predicting user needs anddesires: when a correlation is found for a given user, the actions andneeds of correlated users can be used to suggest actions and presentdata to that user. This can be implemented using a statistical“crowdsourcing” approach as known in the art. When the remotecoordination server has predicted that the user might need or want toperform a particular action or view a particular bit of information, itpresents this to the user. If the user needs to make a decision about aparticular action, the system presents a yes/no dichotomous (ormultiple-choice) query to the user and then performs or does not performthe action depending on the user's response. If the system presents theuser with an unsolicited bit of information, it might ask the userwhether the information was useful or not, or the user might originatevolunteered feedback on such, which the system will note for future. Theremote coordination then records the result of any such query in itshistorical log for future analysis. This intelligent functionality is inaddition to the usual functions of an electronic personal assistantdevice—namely scheduling events, alerts and reminders, and requestingspecific bits of information for display to the user. Augmenting thesebasic organizational functions, the system can keep track of any tasksthe user needs to perform and continue to remind the user of such tasksuntil the user reports their completion or cancellation.

In some embodiments, the queries are not limited to dichotomous answers,but might allow multiple-choice answers as well, giving the user anumber of options to choose from. For example, instead of simply askingfor a yes/no answer, the system might allow a third option, such as “askme later,” in which the user puts off a decision on the matter. Aspecific example of this usage would be a reminder of the user'smother's birthday: the system would remind the user of the birthday andthen offer a set of options, such as (a) send flowers, (b) send candy,(c) send a gift card with a note or (d) do nothing (in which case theuser might handle the gift herself).

Note that in embodiments where more complex input devices are available,such as computer keyboards, the user's input to the system is notlimited to dichotomous or multiple-choice answers, as described below,but can include specific commands, allowing more precise control ofsystem behavior and control of aspects of the system that are difficultor impossible to control using a simple multiple-choice system.

A key portion of the intelligence of the system lies in determining whatdecisions the user might make for a given situation. Rather than simplyasking the user what to do, the system analyzes its database for theuser and all the user's data sources to determine the most likely set ofchoices the user might make, thus “boiling down” or reducing thedecision to a simple set of dichotomous or multiple-choice questions.Further, for more complex decisions, the system would present the userwith a hierarchy of queries, presented as a series, each query narrowingdown the final decision toward an exact action. For example, the systemmight remind the user of a friend's birthday, then ask if he wants topurchase a gift for the friend from the friend's Amazon.com wish list.If the user answers yes, the system would retrieve an item from the wishlist and, perhaps after a delay, ask the user if he wants to purchasethat specific item. If the user answers yes, the system might ask if theuser wants to gift-wrap the item, and then whether the user wants toinclude a message. If the user wants to include a message, the systemmight transcribe it from dictation by the user, or might choose amessage typically sent by the user on such occasions and present it tothe user for approval. Finally, the system would purchase the gift andhave it sent to the friend as specified by the user.

A similar technique can be used for advertising. In some embodiments, ifthe user so chooses, the system can recommend products based on theuser's current circumstances, activities and known data about the user.For example, if the user is known to be shopping for a gift for his wifein a particular location, the system might recommend a specific item hiswife has indicated that she wants, and indicate a nearby store where itcan be purchased. Such as system can further include provisions toprotect the user's privacy by performing all calculations within thesystem and sending none of the user's personal data to outside parties,such as marketers or other companies.

In some embodiments, the system initially requests that the user answeran extensive survey as to various personal and behavioral preferences.The questions of the survey are intended to define key constraints forthe system to ensure its decisions match the user's usual decisions andbehavior as much as possible. Such a survey can include, but is notlimited to, questions such as: What do you give as gifts? How manypeople's birthdays do you celebrate? Do you celebrate birthdays? How doyou remember people? How do you like to stay in contact with people? Doyou write short or long letters? Do you stay in touch with (a)everybody, (b) close friends, (c) relatives? What type of work do youdo? What is your work style? What faith do you hold? What package ofcustoms do you follow? These and other questions give the system aninitial base of behavioral information that will be expanded upon byanalyzing the user's various data stores and sources, and by analyzingthe user's behavior on an historical and ongoing basis. The systemfurther analyzes the user's past communications, such as via email, textmessage, etc., to establish the user's communication style, interests,activities, relationships, personal and business networks, and otherfactors, semantically parsing the communications as is known in the art.The system differentiates between different types of communication(e.g., personal, work, etc.) to establish different categories ofbehavior, relationships and preferences. From this analysis and fromongoing analysis of the user's data and activities, the system infersother factors, including, but not limited to, who the user's friends,family and associates are; how the user prefers to communicate (viaemail, phone, text message, letter, in person, etc.); where the userworks; the user's work style; where the user lives; what the userprefers to do for leisure; where the user likes to vacation; what typesof products the user likes to purchase for himself and others; theuser's religion and degree of religiousness; and the user's sexualpreferences.

Additional “plugin modules” of survey information can be integrated tofurther customize the system to the user. For instance, a number ofmodules may be implemented that connect the system to external datasources, such as LinkedIn™ or Facebook™, email systems, search engineswith browsing history, and numerous other systems as known in the art.By providing a conduit between the rich contextual data about the userprovided by these systems, the present system of this invention caninfer additional patterns of activity and behavior and intent from theuser. Interfacing with such systems through a Web-based API is wellknown in the art.

Additional modules within the system can be developed. For instance, animproved gift-shopping module could be implemented using an API providedby the present invention. This API allows applications to be insertedinto and run within the invention, or to run externally to the systemand access data from within the system, depending on the privacy levelbeing enforced. This improved gift-giving module would aggregate all themost frequently communicated-with people in the user's life, determinewho the user has given gifts to before, aggregate as many birthdays andholiday preferences from the cultural notes about the respective users(e.g., Christmas vs. Hannukah), and determine any other appropriateholidays from address book entries or CRM systems (children's birthdays,etc.). The module would then provide a Web-, mobile-app-,voice-response-, or application-based interview to the user anddetermine what kind of gifts the user wants to be asked about, how many,who, relevant price ranges, etc. The module would then subsequently, incoordination with the present invention, ask the user to approve ordisapprove matching offers. This module could be tied in with a specificcohort of cooperating online retailers or a gifting service.Advantageously, the system could be almost completely automated, but atthe point of choosing a gift, the model could be designed to reduce thegift choice to a human-executable task, using an approach ofAmazon.com's “mechanical turk” system, where the context of therecipient, sender, sending history, price range and a set of options areprovided anonymously to a human gift-chooser. Or, the module couldinterface to a gift service that has been granted non-anonymous accessto the user's gifting patterns, where a human looks at the context andthe overall gifting budget of the user, and designates which giftsshould be given for a set recipients. These choices, whether implementedin the anonymous or non-anonymous approach, are then presented to theuser for approval in accordance with the invention.

Although this approach is herein detailed for the specific case ofgift-giving, modules of this type can be developed by third parties orby the implementers of the invention, and can be implemented in acompletely computer-automated way, using artificial intelligence andstatistical matching techniques, as known in the art, to providepredicted solutions to user needs, or can be used to provide candidateanswers and solutions to service-providing humans via a Web or voiceinterface (as in scheduling/coordination applications), who then usetheir intelligence to solve the critical portion of the solutionproposal. In this way, the system and any integrated or third-partymodules can provide personal assistance in a highly scalable way,because a given expert personal assistant can have his or herdecision-making ability scaled across hundreds of users needingassistance over the same time period. Or, conversely, the personalassistance needed for a given user can be farmed out scalably to humanslocated in disparate locations, providing culturally and contextuallyrelevant solution proposals for the user for a small transactional costas opposed to an hourly or salary rate.

FIG. 6 depicts a possible embodiment of this “plugin module”architecture. The core processing module 602 of the electronic personalassistant system 601 makes available an interface that can be used byauxiliary services. The core delivers events (e.g., birthdays,information requests, meeting times) 603 to the auxiliary service 604,which is designed to interface with the core. The auxiliary service thenresponds with a query or some information 605 to present to the user608. The auxiliary service can either be an external component providedby a third party 604 or an internal component built into the electronicpersonal assistant system 606 (for instance, in customized versions).Further, the auxiliary service can make use of human interaction tosatisfy its functionality 607.

An additional aspect of this intelligent functionality is that thesystem can adapt its level of “presence” or intrusiveness to suit theneeds and desires of the user. For instance, the user can command thesystem to stop presenting queries, reminders, etc., or reduce itsmessage delivery to urgent communication only. In addition, the systemcan detect, based on its historical logs of user activity and itssensory inputs (location sensors, noise sensors, device state, GPS orlatitude/longitude location detected from a mobile device, nearbywireless transmitters, etc.), when the user is in a location or set ofcircumstances requiring less intrusiveness, such as in a businessmeeting or a movie theater, and adjust its level of intrusivenessappropriately. This adaptation can take into account the user's usualschedule, as recorded in the historical logs, and adjust its behavioraccordingly. Further still, the system can query the user as to thedesirability of a particular communication when it is unsureprobabilistically of the current needs of the user as to its level ofintrusiveness, and store the user's response in its historical logs foruse in making future predictions about the user's needs.

In some embodiments, the system initially surveys the user as to hisusual schedule and when he is willing to be interrupted for queries.Such a survey can include, but is not limited to, questions such as: Doyou want to be interrupted (a) in the shower, (b) on the drive to work,(c) when you are sleeping, (d) when your phone is in silent mode, (e)during meetings? For whom do you want to be interrupted? For what sortsof situations do you want to be interrupted? When do you not want to beinterrupted? What are your daily and weekly schedules? Is there aspecific time each day when you would like to be presented withdecisions for the day? Further, the survey can allow the user to specifydegrees of “interruptibility”—that is, during certain times or givencertain circumstances, the user can specify his preferences on a scalefrom “should be interrupted” to “absolutely must not be interrupted.”This can include such designations as “should be interrupted only forimportant matters” and “ask discreetly whether I should be interrupted.”

When the system has a query or information to present to the user, itfirst queries the user as to whether he is willing to be interrupted,unless it is already known that the user is willing to be interrupted(such as at defined times or immediately after another query) or isunwilling to be interrupted. To contact the user, the system uses allavailable communication channels specified by the user for contact, suchas directly via the system's primary interface device, via phone, viaemail, via text message, via instant message, etc. To preserve privacy,if the system does not have a direct line to the user, as via thesystem's primary interface device, it only presents the user with anotification that it is attempting to contact him and does not includeany of the content of the communication. When the user then responds orrequests the remainder of the communication, it is provided via thedesired channel. Further, in some embodiments, when the system requestsan interruption, in addition to allowing or disallowing theinterruption, the user can respond to the request with indication thathe is only to be interrupted if the request is quick, and the systemwill adjudicate the complexity of the communication and withhold it ifit includes more than a specified amount of information or number ofqueries. Thus an interruption is divided into two actions: (1)determining whether the user is willing to be interrupted or notifyingthe user that information or a query is available, and (2) presentingthe query or information.

As it may occur that the user is unreachable at certain times or undercertain circumstances, or simply does not want to answer questions posedby the system, the system can “save up” messages and unansweredquestions and present them to the user when he or she is next available,or when he or she indicates readiness to receive the backlogged messagesand questions. In addition, the system can predict the user'savailability and willingness to receive messages and answer questionsbased on the historical logs of user activity, and adjust itspresentation of information and posing of questions appropriately.Further, the user can define times he wishes to devote to answeringqueries from the system, such as a particular time every day, and thesystem can majorly present non-urgent queries and other information tothe user at the specified time(s).

One use of the present invention is as a planning aid. In someembodiments, the system keeps track of the tasks the user needs or wantsto complete, and when, and helps the user to complete them by making anydecisions it can for the user, and by presenting reminders and queriesto the user at appropriate times. For example, the system might remindthe user to take vitamins daily, walk the dog in the morning, get an oilchange every 3,000 miles of driving or check the status of a networkserver on an hourly basis. The system, having access to all the user'sdata stores, integrates closely with all of the user's calendars andplanning tools, and conforms to any specific planning methodologies usedby the user. For instance, the system can integrate with such tools asMicrosoft Outlook, Google Calendar, Franklin Planner software and othersuch planning tools, and conforms to such planning methodologies asGetting Things Done and the FranklinCovey planning methods, amongothers, as desired by the user. Further, the system includes its ownplanning interface to allow the user to schedule events and tasks to berecorded in any or all of the user's planning tools. In someembodiments, an initial survey of the user is done to establish commontasks and management methods, asking questions such as: What do you needto get done every day, and when? What do you need to be nagged about?What project management do you need? What micromanagement do you need?Which tasks are (a) important, (b) neutral, (c) unimportant, (d) only tobe done in free time? Do you use a specific planning methodology? Do youneed help getting to appointments on time? Do you like to be frequentlyreminded or told only once? Thus, the system gains an initial profile ofthe user's activities and schedules, and can combine this with datagathered from an analysis of the user's data stores. With a knowledge ofall the users schedules and tasks, as well as the planning methodologythe user prefers and specific preferences about reminders, the systemcan proceed to give the user the needed assistance in ensuring schedulesare kept and tasks are carried out in a timely fashion. This isprimarily accomplished by reminding the user of scheduled events andtasks, with a degree of insistence specified by the user on a general ortask-by-task basis, but also includes the other functions of the system,such as recommending events and tasks for scheduling based on analysisof the user's data stores and sources, and partially performingindividual tasks to the greatest degree possible, then presenting simpledichotomous or multiple-choice queries to the user for decision andacting on the decisions to the degree it is possible.

In some embodiments, the system provides the capability of schedulingand initiating meetings automatically. This proceeds roughly as follows:The user indicates that he wants a meeting with a particular person at aparticular time or within a particular range of time. The system thenautonomously attempts to contact the other person to schedule a meeting.It uses all available communication channels—phone, email, text message,instant message, etc.—to contact the other person and then attempts tonegotiate a meeting time within the constraints defined by the user.This may involve repeated attempts to contact the other person, and theuser can adjust the frequency and insistence the system uses in makingsuch repeated attempts to avoid creating a nuisance. If the systemcannot arrive at an agreement about meeting time, it establishes theother person's preferences as to schedule and presents them to the user,who then either chooses an acceptable time or gives a more preferablerange of times. The system may also ask the other party for a scheduleof availability on an ongoing basis for scheduling of future meetings,if it is determined that the person is someone the user is likely tomeet with regularly. This process repeats until an agreement is reached(or aborts if no agreement can be arrived at). Both parties are notifiedof the final agreement, and the system updates the user's calendar andother planning tools to reflect it. Then, when the meeting time nears,the system asks the user if the meeting should proceed, and if so,reminds both parties of the meeting. At the time of the meeting, thesystem attempts to contact the other person via the preferred channelfor the meeting (e.g., phone). Once the other person has been contactedand is on the line, the system contacts the user to join the meeting, ifhe has not joined already, and the meeting proceeds. The system uses asimilar procedure for meetings involving three or more parties, but hasto negotiate with more individuals to arrive at a meeting time.

Similarly, in some embodiments the system has the capability ofattempting to contact others on behalf of the user. When the userindicates that he wishes to speak with a particular person, the systemautonomously attempts to contact the person via any availablecommunication channels, such as phone, email, etc. In some embodiments,the system can even keep track of the other person's behavior,availability and location over time, and use this historical data topredict possible locations and means of contact for the person. If theperson does not respond and is not available (e.g., offline), the systemwatches the various available communication channels (e.g., instantmessenger, VoIP, etc.) to determine when the other person might beavailable. If the user is also known to be available at that time, thesystem attempts to contact the other person and request a meeting. Theseattempts to make contact repeat at some frequency and level ofaggressiveness specified by the user until the other party is eithercontacted or refuses the contact. Once contact is established, thesystem then contacts the user and connects the two parties.

In some embodiments, the system makes use of large stores of historicaluser data for large groups of users (“crowd-sourcing”) to assist indecision-making Where a user's privacy preferences permit, the systemrecords behavioral data for that user and combines it with the storedbehavioral data of all such users to form a behavioral database. Thesystem then analyzes this database to assist in decision-making, usingstandard techniques known in the art for the prediction of preferencesand other choices.

In some embodiments, the user interface described can use a livepersonal assistant service to do the backend work, instead of or inaddition to one or more remote coordination servers. In this case, thesame control interface is presented to the user, but requests for actionare routed as appropriate to the live personal assistant service forfulfillment. Reminders, information and requests from the live personalassistant service would be delivered to the user via the standard userinterface, effectively hiding the existence of the live service andavoiding any qualms the user might have about using such a service dueto concerns about security or other factors.

In some embodiments, the system provides a safety monitoring andnotification service that keeps track of the user's location andexpected path, and sends a warning if something unusual occurs (i.e.,the user's path deviates outside some range of tolerance). For example,the user could specify a starting and ending time and location for awalk or other journey, and if the user failed to arrive in a timelyfashion, the system might warn a family member or the appropriateauthorities. The user could further specify the intended path, and asignificant deviation from that path would trigger a warning.

As configurability is an aspect of the system, it may be necessary toprovide a more complex interface to the user, such as via the Web or viaa software application running on a personal computer or a smartphone.Such an interface would be used for the initial configuration of thesystem, as via a series of survey modules, and could then be used tomake changes to configuration. In some embodiments, configuration wouldbe possible directly via the dichotomous or multiple-choice interfaceitself, as a means of answering many complex questions aboutconfiguration.

A key aspect of the present invention is that it works around thelimitations of many neural, emotional, physical and gesture-based inputmethods by reducing user interaction to the minimum necessary to arriveat a decision. As many such input methods tend to be clumsy, slow andrequire a great deal of concentration, all but the simplest of inputsare too difficult to register. Registering more than a few simple inputsis extremely difficult due to the necessity to differentiate amongpossible stimuli and the degree of processing necessary to do so.Current computing devices and algorithms are simply not up to the task.Thus, the present invention reduces user input to one or moredichotomous or multiple-choice queries that can be answered relativelyeasily using current mobile or hands-free input devices, with a minimumof concentration and a maximum of speed.

Further, the present invention provides real electronic personalassistant functionality, rather than a simple data store and interfacedevice, as it is capable of making decisions on its own, observingsituations relevant to the user and determining the mostly likelyresponses from the user, then presenting the decision to the user forapproval. Thus, the present invention increases the user's efficiencyand reduces the burden on the user by performing as much work aspossible before requesting that the user make a decision.

Thus, a system, method and user interface have been described forproviding a personal assistant functionality using a predictive,adaptive, dichotomous decision engine.

What is claimed is:
 1. A method comprising: receiving a notification ata remote server from an electronics device; analyzing, at the remoteserver, a database having a user data source associated with a user ofthe electronics device to determine a set of choices in response to thenotification, the user data source including information pertaining tothe user selected from one of a web-based email, a contact database,data stored on a social networking web-site, an online retailtransaction, and combinations thereof, the set of choices includingassociated actions that the user is predicted to take in relation to thenotification; creating, at the remote server, a query based on the setof choices, the query including dichotomous or multiple choice questionsarranged in a hierarchical structure to select among the set of choices,the hierarchical structure having a plurality of levels, where a toplevel of the hierarchical structure corresponds to at least one of theset of choices, and lower levels of the hierarchical structurecorrespond to the associated actions that the user is predicted to takein relation to the notification; sending the query to the electronicsdevice for presentation to the user; receiving, from the electronicsdevice at the remote server, answers in response to a plurality of thedichotomous or multiple choice questions; traversing, at the remoteserver, the hierarchical structure, based on the answers in response tothe plurality of the dichotomous or multiple choice questions, to reducethe set of choices to a first choice from the set of choices and toidentify a first associated action that the user is predicted to takewhere the first associated action is associated with the first choice;and performing the first associated action, thereby to improve operationof the remote server to predict and implement a selection from the setof choices.
 2. The method of claim 1 wherein the database includes ahistorical log of actions selected by the user.
 3. The method of claim 1further comprising adapting a future query based on one or moreadditional responses to one or more earlier queries.
 4. The method ofclaim 3 wherein the future query is used to determine whetherunsolicited information in an earlier query is useful.
 5. The method ofclaim 1 wherein the user data source is analyzed based on a currentlocation of the user.
 6. The method of claim 1 wherein the query sent tothe electronics device is responded to using a neural input deviceresponsive to physical movements of the user.
 7. The method of claim 1wherein the performed first associated action augments organization ofthe associated actions by the electronics device.
 8. The method of claim1 wherein sending the query includes sending the user a sequence ofqueries, each query successively narrowing the selection made by theuser for performing the first associated actions.
 9. The method of claim1 wherein one of the dichotomous or multiple choice questions permitsthe user to postpone performing the first associated action.
 10. Acomputer program product comprising non-transitory computer executablecode embodied in a non-transitory computer readable medium configuredto: receive a notification at a remote server from an electronicsdevice; analyze, at the remote server, a database having a user datasource associated with a user of the electronics device to determine aset of choices in response to the notification, the user data sourceincluding information pertaining to the user selected from one of aweb-based email, a contact database, data stored on a social networkingweb-site, an online retail transaction, and combinations thereof, theset of choices including associated actions that the user is predictedto take in relation to the notification; create, at the remote server, aquery based on the set of choices, the query including dichotomous ormultiple choice questions arranged in a hierarchical structure to selectamong the set of choices, the hierarchical structure having a pluralityof levels, where a top level of the hierarchical structure correspondsto at least one of the set of choices, and lower levels of thehierarchical structure correspond to the associated actions that theuser is predicted to take in relation to the notification; send thequery to the electronics device for presentation to the user; receive,from the electronics device at the remote server, answers in response toa plurality of the dichotomous or multiple choice questions; traverse,at the remote server, the hierarchical structure, based on the answersin response to the plurality of the dichotomous or multiple choicequestions, to reduce the set of choices to a first choice from the setof choices and to identify a first associated action that the user ispredicted to take where the first associated action is associated withthe first choice; and perform the first associated action, thereby toimprove operation of the remote server to predict and implement aselection from the set of choices.
 11. The computer program product ofclaim 10 wherein the database includes a historical log of actionsselected by the user.
 12. The computer program product of claim 10wherein the computer executable code is further configured to adapt afuture query based on one or more additional responses to one or moreearlier queries.
 13. The computer program product of claim 12 whereinthe future query is used to determine whether unsolicited information inan earlier query is useful.
 14. The computer program product of claim 10wherein the user data source is analyzed based on a current location ofthe user.
 15. The computer program product of claim 10 wherein the querysent to the electronics device is responded to using a neural inputdevice responsive to physical movements of the user.
 16. The computerprogram product of claim 10 wherein the performed first associatedaction augments organization of the associated actions using theelectronics device.
 17. The computer program product of claim 10 whereinsending the query includes sending the user a sequence of queries, eachquery successively narrowing the selection made by the user forperforming the first associated action.
 18. The computer program productof claim 10 wherein one of the dichotomous or multiple choice questionspermits the user to postpone performing the first associated action. 19.A system comprising: a processor and a memory connected to acommunications interface for establishing communication with anelectronics device, the processor executing a task configured to receivea notification from the electronics device, analyze a database having auser data source associated with a user of the electronics device todetermine a set of choices in response to the notification, the userdata source including information pertaining to the user selected fromone of a web-based email, a contact database, data stored on a socialnetworking web-site, an online retail transaction, and combinationsthereof, the set of choices including associated actions that the useris predicted to take in relation to the notification, create a querybased on the set of choices, the query including dichotomous or multiplechoice questions arranged in a hierarchical structure to select amongthe set of choices, the hierarchical structure having a plurality oflevels, where a top level of the hierarchical structure corresponds toat least one of the set of choices, and lower levels of the hierarchicalstructure correspond to the associated actions that the user ispredicted to take in relation to the notification, send the query to theelectronics device for presentation to the user, receive answers inresponse to a plurality of the dichotomous or multiple choice questions,traverse the hierarchical structure, based on the answers in response tothe plurality of the dichotomous or multiple choice questions, to reducethe set of choices to a first choice from the set of choices and toidentify a first associated action that the user is predicted to takewhere the first associated action is associated with the first choice,and perform the first associated action, thereby to improve operation ofthe processor executing the task to predict and implement a selectionfrom the set of choices.
 20. The system of claim 19 wherein the databaseincludes a historical log of actions selected by the user.